add meta code verify #311

spiritbroski · 2023-03-09T12:14:41Z

closes #277

Video Demo

https://www.youtube.com/watch?v=BZfnLPGep-4

Description

This is a modified version of https://github.com/facebookincubator/meta-code-verify by facebook, to describe what it does in simple terms,it is basically a web extension for checking if the js doesnt get tampered by creating a merkle tree both in a client and in a server you can read this https://wiki.mozilla.org/Security/Binary_Transparency, if you see in the code that i push you can see that i create a scripts https://github.com/spiritbroski/human-protocol/blob/24b39697a51096c9f982b00b44a469b0c11470de/packages/apps/escrow-dashboard/scripts/generateMerkleTree.ts to generate merkle tree in vite then push the root of the merkle tree to ipfs, i design it to look like the whatsapp binary transparency manifest:

this is what it look like in our apps:

the difference is i remove hash_function , and for the whatsapp they push their merkle root into cloudflare like this https://staging-api.privacy-auditability.cloudflare.com/v1/hash/whatsapp.com/2.2313.8 and this is ours https://bafkreicgn4acvov2oilxnbeootwvxdnakn4a6lcf3sgs2ubi5u76wcdrny.ipfs.nftstorage.link/ its the same and we are much more resilient coz its decentralized in ipfs however cloudflare is centralized server, so this is a brief description of what i've been working on

How to use

As this is a web extension first you need to have either chrome based browser or firefox, in this demo i use brave browser, first navigate to packages/apps/meta-code-verify and run this script:

$ yarn
$ yarn build-local-dev

Then go to your browser and open this url brave://extensions/ turn on developer mode if you haven't:

Then click load unpacked:

Then navigate to the dist/chrome folder of meta-code-verify then click select folder

If successful it will show something like this:

Now go to packages/apps/escrow-dashboard and run this command:

$ yarn
$ yarn build --mode development

Wait until it finish building then run

$ yarn start-prod

Go to your browser and navigate to http://localhost:3000, if you click on the web extension earlier you can see the green checkmark which mean our code is not tampered

So how to check if our code is getting tampered first we need to stop the yarn start-prod command if youre on linux its simply CTRL+C then now go to index.html in escrow-dashboard packages and add this <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.3/jquery.min.js"></script>:

and run yarn build --mode development again and followed by yarn start-prod and when you go to http://localhost:3000 again you will see red exclamation mark:

It means that one or more script is not in the merkle tree and if you download it you will get list of all js files and its source code in gzip files so you can check it yourself, ok thats it waiting for your review

CLAassistant · 2023-03-09T12:14:48Z

All committers have signed the CLA.

vercel · 2023-03-09T12:14:49Z

@spiritupbro is attempting to deploy a commit to the HUMAN Protocol Team on Vercel.

A member of the Team first needs to authorize it.

leric7

Target branch should be develop

packages/apps/meta-code-verify/yarn.lock

packages/apps/meta-code-verify/.github/workflows/tests.js.yml

packages/apps/meta-code-verify/CODE_OF_CONDUCT.md

packages/apps/meta-code-verify/CONTRIBUTING.md

packages/apps/meta-code-verify/LICENSE.md

packages/apps/meta-code-verify/README.md

packages/apps/meta-code-verify/yarn.lock

portuu3

I get this error when I run yarn build-local-dev

spiritbroski · 2023-04-06T07:56:03Z

Gotchu let me try cloning fresh and then get back to you

…

On Thu, Apr 6, 2023, 2:54 PM portuu3 ***@***.***> wrote: ***@***.**** requested changes on this pull request. I get this error when I run yarn build-local-dev [image: image] <https://user-images.githubusercontent.com/61605646/230311318-42ff20e5-42b7-4f0c-9af7-1442952e1a47.png> — Reply to this email directly, view it on GitHub <#311 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO5B4AKQ4S7SB6QIFSJOE4DW7ZZE7ANCNFSM6AAAAAAVVBCPFE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

spiritbroski · 2023-04-06T08:06:43Z

@portuu3 please pull again and try again it turns out 'build' not getting pushed it basically a script for rollup, it works now

portuu3 · 2023-04-11T13:15:32Z

@portuu3 please pull again and try again it turns out 'build' not getting pushed it basically a script for rollup, it works now

I am getting this now when I run yarn build --mode development:

spiritbroski · 2023-04-11T13:17:51Z

ah its need nft.storage token i need fix the docs i guess 😅 sorry let me fix the docs you need to add ` VITE_APP_NFT_STORAGE_API= `

…

On Tue, Apr 11, 2023 at 8:15 PM portuu3 ***@***.***> wrote: @portuu3 <https://github.com/portuu3> please pull again and try again it turns out 'build' not getting pushed it basically a script for rollup, it works now I am getting this now when I run yarn build --mode development: [image: image] <https://user-images.githubusercontent.com/61605646/231174169-491c2ed9-1efe-4460-8e5e-0bb0a5938180.png> — Reply to this email directly, view it on GitHub <#311 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO5B4AJ3YYSIQG6BDP6VWTDXAVKQBANCNFSM6AAAAAAVVBCPFE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

portuu3 · 2023-04-13T07:10:18Z

Now if builds ok for both code verify and dashboard, but I get this without modifying index.html

spiritbroski · 2023-04-13T07:11:28Z

Is this on localhost?

…

On Thu, Apr 13, 2023, 2:10 PM portuu3 ***@***.***> wrote: Now if builds ok for both code verify and dashboard, but I get this without modifying index.html [image: image] <https://user-images.githubusercontent.com/61605646/231681182-145bbe8e-cfdb-4335-b2af-c79244acb58c.png> — Reply to this email directly, view it on GitHub <#311 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO5B4APXEQZYFQO2GGUTP6TXA6RGLANCNFSM6AAAAAAVVBCPFE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

spiritbroski · 2023-04-13T07:15:35Z

Oh wait @portuu3 could you please try using incognito? Because in the original meta-code-verify by facebook there are a list of whitelist extension if there are a blacklisted extension it will trigger validation warning coz sometimes browser extension also inject a script see this one facebookincubator/meta-code-verify#139 i think i need to fix the docs again

mrhouzlane · 2023-04-22T03:18:38Z

https://github.com/spiritbroski/human-protocol/blob/meta-code-verify/packages/apps/escrow-dashboard/README.md#yarn-run-build

For me extension is not showing up for http://localhost:3000/, but showing for https://web.whatsapp.com/.
I have added incognito mode and load unpacked. Have tried the extension both on Brave and Chrome. Same result.

Localhost

spiritbroski · 2023-04-22T03:23:43Z

@mrhouzlane can you tell me how you run this? Are you using this readme https://github.com/spiritbroski/human-protocol/blob/meta-code-verify/packages/apps/meta-code-verify/README.md to run? i fresh clone the repo and it work for me this is my reproduction step:

clone meta code verify branch
go to packages/apps/escrow-dashboard adding necessary .env run yarn then yarn start-prod
go to packages/apps/meta-code-verify run yarn then yarn build-local-dev
Load unpacked packages/apps/meta-code-verify/dist/chrome in brave browser
go to localhost:3000 and then i got validated screen i dont have any other extension installed for this reproduction step

are you also run it just like me? and are you adding something in index.html? have you tried refreshing it, is it the same? could you please disable any other extension other than meta code verify first then try reload again?

mrhouzlane · 2023-04-22T11:39:38Z

@mrhouzlane can you tell me how you run this? Are you using this readme https://github.com/spiritbroski/human-protocol/blob/meta-code-verify/packages/apps/meta-code-verify/README.md to run? i fresh clone the repo and it work for me this is my reproduction step:

clone meta code verify branch

go to packages/apps/escrow-dashboard adding necessary .env run yarn then yarn start-prod

go to packages/apps/meta-code-verify run yarn then yarn build-local-dev

Load unpacked packages/apps/meta-code-verify/dist/chrome in brave browser

go to localhost:3000 and then i got validated screen i dont have any other extension installed for this reproduction step

are you also run it just like me? and are you adding something in index.html? have you tried refreshing it, is it the same? could you please disable any other extension other than meta code verify first then try reload again?

Working now.
If you can those steps in readme would be good.

Modify the 2.
2. go to packages/apps/escrow-dashboard adding necessary .env run yarn build --mode development then then yarn start-prod

spiritbroski · 2023-04-22T11:42:18Z

@mrhouzlane can you tell me how you run this? Are you using this readme https://github.com/spiritbroski/human-protocol/blob/meta-code-verify/packages/apps/meta-code-verify/README.md to run? i fresh clone the repo and it work for me this is my reproduction step:

clone meta code verify branch

go to packages/apps/escrow-dashboard adding necessary .env run yarn then yarn start-prod

go to packages/apps/meta-code-verify run yarn then yarn build-local-dev

Load unpacked packages/apps/meta-code-verify/dist/chrome in brave browser

go to localhost:3000 and then i got validated screen i dont have any other extension installed for this reproduction step

are you also run it just like me? and are you adding something in index.html? have you tried refreshing it, is it the same? could you please disable any other extension other than meta code verify first then try reload again?

Working now. If you can those steps in readme would be good.

Modify the 2. 2. go to packages/apps/escrow-dashboard adding necessary .env run yarn build --mode development then then yarn start-prod

@mrhouzlane which readme? escrow-dashboard or meta-code-verify? also could you please try using metamask on? is it throwing error? if it is then my app is correct, also use https://github.com/spiritbroski/human-protocol/tree/meta-code-verify/packages/apps/meta-code-verify#allowlist to allow to use metamask without disabling it first

mrhouzlane · 2023-04-22T11:45:06Z

@mrhouzlane can you tell me how you run this? Are you using this readme https://github.com/spiritbroski/human-protocol/blob/meta-code-verify/packages/apps/meta-code-verify/README.md to run? i fresh clone the repo and it work for me this is my reproduction step:

clone meta code verify branch

go to packages/apps/escrow-dashboard adding necessary .env run yarn then yarn start-prod

go to packages/apps/meta-code-verify run yarn then yarn build-local-dev

Load unpacked packages/apps/meta-code-verify/dist/chrome in brave browser

go to localhost:3000 and then i got validated screen i dont have any other extension installed for this reproduction step

are you also run it just like me? and are you adding something in index.html? have you tried refreshing it, is it the same? could you please disable any other extension other than meta code verify first then try reload again?

Working now. If you can those steps in readme would be good.
Modify the 2. 2. go to packages/apps/escrow-dashboard adding necessary .env run yarn build --mode development then then yarn start-prod

@mrhouzlane which readme? escrow-dashboard or meta-code-verify?

Here : https://github.com/spiritbroski/human-protocol/tree/meta-code-verify/packages/apps/meta-code-verify

Steps are already detailed, all good.
Removing other extensions I think made it work.

Do you have any tests for generateMerkleTree function ?

spiritbroski · 2023-04-22T12:09:15Z

Removing other extensions I think made it work.

uhm, no basically the correct flow if there is any other extension it will throw an error or a risk message so thats why i ask you that, and thats why i create allowlist so you can allow some of the extension that you deemed safe to use, you can try it like this install metamask and go to localhost:3000 it will throw validation error

Do you have any tests for generateMerkleTree function ?

nope, i will add that i guess

spiritbroski · 2023-04-22T14:17:23Z

@mrhouzlane adding the generateMerkleTree test check again

posix4e

looks good on first view but i'll leave it to others to approve

mrhouzlane · 2023-04-23T05:28:17Z

@mrhouzlane adding the generateMerkleTree test check again

Test working, good for me.

posix4e · 2023-04-23T13:36:32Z

Let's fix the lint and tests

spiritbroski · 2023-04-23T20:18:47Z

@posix4e ok

spiritbroski · 2023-04-23T23:36:45Z

@posix4e i think most of the check failing is not coming from the package that i work with

portuu3 · 2023-04-24T07:00:45Z

@posix4e i think most of the check failing is not coming from the package that i work with

Merge develop branch and check if it gets fixed

spiritbroski · 2023-04-24T07:11:40Z

Ok let me try that

…

On Mon, Apr 24, 2023, 2:00 PM portuu3 ***@***.***> wrote: @posix4e <https://github.com/posix4e> i think most of the check failing is not coming from the package that i work with Merge develop branch and check if it gets fixed — Reply to this email directly, view it on GitHub <#311 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO5B4AN4XN4SEQU2ADV2VGLXCYQKTANCNFSM6AAAAAAVVBCPFE> . You are receiving this because you were assigned.Message ID: ***@***.***>

spiritbroski · 2023-04-24T07:27:37Z

@portuu3 it said already up to date

Changes required made

Add meta code verify to check integrity of dashboard

first commit

a0ab233

spiritbroski added 3 commits March 27, 2023 20:09

Merge branch 'main' into meta-code-verify

41fde68

done and need review

5d311ab

add all of em

24b3969

spiritbroski marked this pull request as ready for review March 28, 2023 10:45

spiritbroski added 3 commits March 28, 2023 18:56

fix lint

3c92c59

adding origin to ipfs, also check whether origin is the same

ed25046

fix lint

f008c94

leric7 reviewed Apr 4, 2023

View reviewed changes

packages/apps/meta-code-verify/yarn.lock Outdated Show resolved Hide resolved

packages/apps/meta-code-verify/.github/workflows/tests.js.yml Outdated Show resolved Hide resolved

spiritbroski changed the base branch from main to develop April 4, 2023 13:19

spiritbroski added 2 commits April 4, 2023 20:24

remove yarn.lock and put test inside .github root folder

2fc75cf

Merge branch 'develop' into meta-code-verify

291a13d

spiritbroski requested a review from leric7 April 4, 2023 13:35

portuu3 requested changes Apr 5, 2023

View reviewed changes

add README.md and delete unnecessary files

45f9736

spiritbroski requested a review from portuu3 April 5, 2023 08:28

Merge branch 'develop' into meta-code-verify

45c9746

portuu3 requested changes Apr 6, 2023

View reviewed changes

rename build to built

15f2cc0

spiritbroski requested a review from portuu3 April 6, 2023 08:07

fix the docs

f206b22

adding generateMerkleTree.ts test

920abfe

posix4e approved these changes Apr 22, 2023

View reviewed changes

posix4e self-requested a review April 22, 2023 14:17

posix4e assigned spiritbroski Apr 22, 2023

posix4e removed their request for review April 22, 2023 14:17

vercel bot temporarily deployed to Preview – fortune-exchange-oracle-server April 22, 2023 14:18 Inactive

posix4e previously requested changes Apr 22, 2023

View reviewed changes

fix lint

49816a2

vercel bot temporarily deployed to Preview – fortune-exchange-oracle-server April 23, 2023 20:32 Inactive

Merge branch 'develop' into meta-code-verify

4fbf7fd

vercel bot temporarily deployed to Preview – fortune-exchange-oracle-server April 25, 2023 01:54 Inactive

portuu3 approved these changes Apr 25, 2023

View reviewed changes

portuu3 merged commit 12c9e7c into humanprotocol:develop Apr 25, 2023

m00n620 pushed a commit that referenced this pull request May 4, 2023

add meta code verify (#311)

7a90acf

Add meta code verify to check integrity of dashboard

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add meta code verify #311

add meta code verify #311

spiritbroski commented Mar 9, 2023 •

edited

Loading

CLAassistant commented Mar 9, 2023 •

edited

Loading

vercel bot commented Mar 9, 2023

leric7 left a comment

portuu3 left a comment

spiritbroski commented Apr 6, 2023 via email

spiritbroski commented Apr 6, 2023

portuu3 commented Apr 11, 2023

spiritbroski commented Apr 11, 2023 via email

portuu3 commented Apr 13, 2023

spiritbroski commented Apr 13, 2023 via email

spiritbroski commented Apr 13, 2023

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 •

edited

Loading

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 •

edited

Loading

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 •

edited

Loading

spiritbroski commented Apr 22, 2023

posix4e left a comment

mrhouzlane commented Apr 23, 2023

posix4e commented Apr 23, 2023

spiritbroski commented Apr 23, 2023

spiritbroski commented Apr 23, 2023

portuu3 commented Apr 24, 2023

spiritbroski commented Apr 24, 2023 via email

spiritbroski commented Apr 24, 2023

add meta code verify #311

add meta code verify #311

Conversation

spiritbroski commented Mar 9, 2023 • edited Loading

Video Demo

Description

How to use

CLAassistant commented Mar 9, 2023 • edited Loading

vercel bot commented Mar 9, 2023

leric7 left a comment

Choose a reason for hiding this comment

portuu3 left a comment

Choose a reason for hiding this comment

spiritbroski commented Apr 6, 2023 via email

spiritbroski commented Apr 6, 2023

portuu3 commented Apr 11, 2023

spiritbroski commented Apr 11, 2023 via email

portuu3 commented Apr 13, 2023

spiritbroski commented Apr 13, 2023 via email

spiritbroski commented Apr 13, 2023

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 • edited Loading

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 • edited Loading

mrhouzlane commented Apr 22, 2023

spiritbroski commented Apr 22, 2023 • edited Loading

spiritbroski commented Apr 22, 2023

posix4e left a comment

Choose a reason for hiding this comment

mrhouzlane commented Apr 23, 2023

posix4e commented Apr 23, 2023

spiritbroski commented Apr 23, 2023

spiritbroski commented Apr 23, 2023

portuu3 commented Apr 24, 2023

spiritbroski commented Apr 24, 2023 via email

spiritbroski commented Apr 24, 2023

spiritbroski commented Mar 9, 2023 •

edited

Loading

CLAassistant commented Mar 9, 2023 •

edited

Loading

spiritbroski commented Apr 22, 2023 •

edited

Loading

spiritbroski commented Apr 22, 2023 •

edited

Loading

spiritbroski commented Apr 22, 2023 •

edited

Loading